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ABSTRACT 

Research, evaluation, and policy analysis are 
eleaents of inquiry whose functions, aias, purposes, intended 
audiences, and intended outcones have been confused in the literature 
discussing how to accoaplish thea. Using the definition of 
"disciplined inquiry* provided by Cronbach and Suppes (1969), which 
defines disciplined inquiry as the resolution of a problea to achieve 
understanding or to facilitate action, an analysis is nade of each 
research, evaluation, and policy analysis. Each is found to fit 
within the definition of disciplined inquiry, and then each is 
dissected to conpare purposes, audiences, and intended outcones. 
Proposed for each is a new definition that incorporates the fonts, 
functions, audiences, and outcomes. A bibliography is included. 
(Author/TE) 
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ABSJRACT 

Research, evaluation and policy analysis are elements of 
inquiry whose functions, aims, purposes, intended audiences and 
intended outcomes have been confused in the literature regarding 
how to accomplish them. Using the definition of disciplined 
inquiry provided by Cronbach and Suppes (1969), an analysis is 
made of each, each is found to fit within the definition, and 
then each is dissected to compare purposes, audiences And intended 
outcomes. New definitions are proposed for each which incorporate 
the forms, functions, audiences ;ind out^'omes. 



Research, Evaluation, and Policy Analysis : 

Heuristics for Disciplined Inquiry ^ 

If hanging loose is our national genius, then we social scientists 
should have more tolerance for what passes as Inefficiency and 
non-rationality. The argument would run like this: People should 
continue to be intendedly rational; there is nothing wrong with 
rationality per se . However, given human limits, people should 
avoid constructing systems— social and engineering and ideological ^ 
systems— that exceed human rationality (Perrow, 1981). 



The Proliferation— and Confusion-7of Information Specialities 
We live in a world of finite resources with a seemingly infinite 
number of Interest groups all seeking to exploit them. It is not 
surprising, therefore, that demands for better and more complete 
information needed to service decision making, policy formation, 
and resource allocation are burgeoning. Persons from a wide 
variety of disclplines--political science, education, sociology, 
psychology, management, to name a few— are making efforts io 
respond to these information needs. But as each group brings its 
own disciplinary perspective to bear on the problem, the resulting 
proliferation of concepts and terms tends to confuse rather than to 
clarify the state of affairs. 

The need to maintain status and legitimation in the home 
discipline also plays an important role; thus, both evaluation and 
policy analysis--"applied" areas clearly on the periphery of the 
more "pure" or basic discipllne--acquire a patina of respectability 
as academic products or activities by the simple addition of the 
word "research." Evaluation becomes "evaluation research" (Suchman, 
1967; Mann, 1972; Caro, 1971), and policy analysis becomes "policy 
analysis research" (Carley, 1980, p. 14). As a consequence, 
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whether deliberate or inadvertent^ the distinctions between .research, 
evaluation, and policy analysis have beconvs blurred. 

The blurring of distinctions has produced little clarity and a 
good bit more heat than light. The mixed terminology, coupled 
with a lack of commonly accepted definitions, has led to confusion 
regarding the aims of research, evaluation, and policy analysis, 
the types of products or outcomes consumers have a right to 
jexpect when each of these activities Is undertaken, and, for 
academicians, the legitimacy of each as a "scholarly" activity. 
Furthermore, the great paradigm debate in the social sciences 
suggests that definition development alone may not be enough. 
Some consideration of paradigmatic underpinnings of inquiry may 
also be called for, at least to suggest where choice of paradigm 
might prove crucial in the outcomes of inquiry efforts. 

This state of affairs suggests that a critical analysis of how 
these activities are defined and used bv^ t practitioners, whether 
in colleges and universities, state or fede. «« agencies, or "on the 
firing line," might be useful in relation to such questions as 
these: 

*'How do these activities relate to each other, if at all? 

**Are all these activities some form of research (as their mixed 
nomenclature might lead us to believe), or do they exemplify 
different forms of knowledge creation, transformatipn, or 
utilization activities? 

**What are the legitimate purposes (outcomes) of each? 

**Who are the Intended audiences for each? 

**Are they well served by conventional scientific inquiry or 
might they be better served by emergent, alternative or 
naturalistic paradigm? 

It is the purpose of this work to deal with each of these questions. 

5 



3 



Are Research^ Evaluation/ and Policy Analysis Related ? 

It seems clear that the use of such hybrid terms as "evaluation 

research" (or' the unbelievable term recently encountered in the 

literature, "policy analysis evaluation research") has little to 

recommend it. Yet the three terms are not entirely independent 

either. We do not mean to denote, by that observation, the fact 

that the same tools— methods— are used in each; to assert identity 

or similarity on the basis of common methods would be analogous to 

saying that carpenters, electricians, and plumbers do the same 

thing because their tool kits all contain hammers, saws, wrenches, 

and screwdrivers. Similarity is rather asserted on the grounds 

that all three are variants of what is commonly called "disciplined 

inquiry." Cronbach and Suppes (1969) suggest that the report of 

a disciplined Inquiry 

. . . has a texture that displays the raw materials 
entering Into the argument and the logical processes by 
which they were compressed and rearranged to make the 
conclusion credible (pp. 15-16). 

That is, to qualify as disciplined, the report of an inquiry must 

inform the reader, in ways that are publicly confirmable, what the 

r" 

nature of the "raw" data is, the sources of those data, and the 
context in which they were collected (for example, a laboratory, 
the respondents' work places, and the like). At the same time, 
the processes for transforming the data into information--lnterpreta- 
tions, conclusions, extrapolations, recommendations- -must also be 
apparent to the reader; they too must be publicly confirmable so 
that their logic and coherence can be tested. 

Now there is no question that not all reports of research, 
evaluations, or policy analyses conform to these requirements - 
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But It is equally evident that there is no reason in principle that 
prevents any of these modes from qualifying as disciplined, if they 
have been properly carried out. The reader should be In a position 
to answer each of the following questions with "Yes": Are the raw 
materials clearly displayed? Do I understand the logic by which 
the data were reorganized into the argament? Does the argument 
exhibit logic and coherence? Clearly, most exemplars of research, 
evaluation, and policy analysis do lead to "Yes" answers. 

But it does not follow from that assertion that these activities 
are identical or interchangeable. Indeed, it is possible to differen- 
tiate them along a number of dimensions, particularly their purposes, 
products or outcomes, and their audiences. The following three 
sections consider each activity in turn. 

Research . 

What is a generally acceptable definition of research? Webster's 
New Collegiate Dictionary offers "studious inquiry" as- a first 

definition, presumably to. differentiate it from casual inquiry. 

J- 

Provu^ (cited In Gephart, Ingle, and Saretsky, 1973) suggests 
that research Is characterized by three dimensions: "1) it is 
problem oriented, 2) it is a systematic process, and 3) it is objective 
(in the sense that it is] free of introduced for unaccounted for] 
bias, (and) It uses empirical evidence" (p. IT). But that seems to 
be simply another way of saying that research is disciplined inquiry, 
without providing any insight into what form of disciplined Inquiry 
It might be. Gephart, Ingle and Saretsky (1973) suggest that the 
aim of research is creating "generalizable knowledge; ... to fmd 
out, in one sense, truth" (p. 11). At the same tim«f they comment 



that "a specific or practical outcome is usually not the major goal 
of this type of research although it is usually possible to infef^ 
some application" (p. 11). Thus the common distinction between 
basic and applied research is maintained. - 

That distinction is not very useful, however; it simply reflects 
the need of some scientists to claim superiority over others. This 
belief Is perhaps best demonstrated by the observation that basic 
research is ofted called "pure" research, evidently to distinguish 
it from its less pure cousin. Attempts to differentiate these two 
putatively different froms of research on the basis that the problems 
they address come from different sources (one is said to derive 
from the innate curiosity of the researcher the other is 

defined by a client group) seem unpersuasive. Basic research is, 
after all, sometimes mounted in applied settings because that is 
where the necessary resources, facilities, or "subjects" are to be 
found. Similarly, the suggestion that the purposes of these two 
forms are different- -theory development versus problem solving or 
responding to a need--is not helpful. Applied research often - 
produces the "facts" that challenge an entrenched theory. Further^ 
the results of applied research may have theoretical utility, as for 
example, when data on the creep of metals under stress, intended 
to inform engineers about how structural members of bridges or 
buildings deform over time because of the loads they bear, may 
also provide insights into the nature of nwlecular movement. 
Indeed, It seems possible to assert that many basic advances in 
the sciences have been occasioned by efforts to solve practical 
problems. 
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Nor do audience dj$tjnctiofTS--pure scientists versus cli?nt 
•s groups (which may ihclude applied peers^ e.g. , fellow engineers)— 
seem useful. Laymen may be interested in theory and scientists in 
applications. i 

A further major difficulty with extant definitions of research 
Is that virtually all are cast within a positivist framework. Thus 
the purposes of research are said to be prediction and control, 
through explanation in the erklarung sense; the major products 
are seen as technical reports that delineate variables and specify 
relationships among them (preferably causal); the major audiences 
are seen as one's scientific peers and do not include those whose 
characteristics, behaviors, values, attitudes, and the like-'that is, 
the "subjects" --are represented in the research. Those social 
scientists who are increasingly finding themselves disenchanted 
with the results of research couched in positivist terms, and who 
are therefore, drawn to naturalist approaches, find little to attract 
them in these conventional definitions. A more open view of 
research would alios ail inquirers the freedom to operate on either 
traditional or emergent philosophical grounds. 

We believe that the following definition of research has much 
to recommend it, both in that it avoids the basic/applied dilemma 



and because i^t is congenial to those with an emeroerit-paradigm or 



naturalist perspective, while not closing out tne scientific or 

rationalistic position: ^ 

RESEARCH Is a type of DISCIPLINED INQUIRY undertaken 
to resolve some PROBLEM in order to ACHIEVE UNDER- 
STANDING or to FACILITATE ACTION. 

This definition, first asserts that research Is disciplined 

Inquiry, which means that we commit It both to publicly examinable 
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^nd verifiable "raw materials'' (data) and '•compression and rearrange- 
ment" processes, as* specified tiy Cronbach artd Suppes (1969). We^ 
have elsewHIere made the case thai^ naturalistic research fulfills 
these conditions as well or better than conventional research (Guba 
and Lincoln, 1983, Lincoln and Guba, in press). 

Second, the definition asserts that the focus of research is a 
problenp . The outcome of research is some problem resolution or 
amelioration (whether theoretical or practical In nature), a formulation 
acceptable within both paradigms. This statement suggests that 
conventional and naturalistic inquiry both bound inquiry in exactly ^ 
the same way; one Is no more "empty -headed" than the other. 
Further, since problems can be of various types (Guba and Lincoln,, 
1981), including conceptual, action, and value problems, naturalists 
can note with satisfaction that the value dimension is provided for. 

Third, the definition averts that the outcome of research Is' 
either the achievement of understanding or the facilitation of action 
or both. Understanding can mean, if one wishes it to, prediction, 
control, and erklarung , bqt it can equally well mean description, 
elucidation of meaning, or verstehen . Both positivists and naturalists 
can find a home within one or the other of these interpretations. 
Finally, the inclusion of the phrase "facilitation of action" implies 
that what is conventionally known as "applied" research Is neverthe- 
less recognized as fully legitimate. 

If this furnishes a det 'nition of research, then how may 
evaluation be defined? 
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Evaluation 

There Is no general agreement among the practitioners of the 

f ^ • 

art/science of evaruation at>out what evaluation is. Evaluation is 

variQMsiy defined as: 

**A proces^ for determining congruence of performance with 
objectives or intents (Tyler, 1949; Provus, 1971; Popham, 
1975). . 



®A process of delineating, obtaining and providing useful 
t information for judging decision alternatives (Stuff iebeam, 
1971). 

**A process fi^r comparing actual effects td a profile of demon- 
strated needs (Scriven, 1973). 

**A process for critically describing and appraising an evaluand 
throu^fh connoisseurship and criticism (Eisner, 1979). 

Building especially on the insights of "responsive" evaluators*. 
(Stake, 1975; Parlett and Hamilton, ^72; Patton, 1980), we have 
suggested (Guba and Lincoln, 1981; Guba and Lincoln, 1983) that 
there are four different types of evaluation, generated by crossing 
Scriven's (1967) well known dimensions formaTKinfe/sulnmajtive with . ' 
two aspects of value that an eva.uator may seek. to ^Ush: - 
mcirlt and worth . Merit, as we have construed it, refers to a -kind 
of "intrinsic, context-free value** illustrated, vfor example, by the 
schoiarllness of a professor (the qualities on which promotion is 
based) or the sequence, continuity, and Integration of a curriculum. 
These are qualities that accompany the evaluand from context to 
context and are relatively invariant. Worth, on the other hand, 
refers to an "extrinsic, context-determined value" examplified by 
the professor's utility vo the institution that employs him (the 
qualities on which tenure is based),* or the utility of the curriculum 
for teaching a certain kind of student in a certain setting. These 
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are qualities that are at least as much dependent on context as on 
the evalu£tnd) while merit is relatively invariant worth varies 
greatly from context to context. 

The terms formative and summative refer, broadly speaking, 
to the aims of evaluation (Scriven calls them roles). The aim of 
formative evaluation \i to provide descriptive and judgmental 
information, leading to refinement. Improvement, alterations, and/or 
modification in the evaluand, while the aim of summative evaluation 
is to determine its impacts, outcomes, or results. 

The dimensions formative/summative and merit/worth may be 
taken as orthogonal to one another to generate the four types of 
evaluation described (Cuba and Lincoln, 1981). Thus, formative merit 
evaluation Is performed to modify or improve some evaluand while it 
is' in process of development . Summative merit evaluation is performed 
In or6er to certify or warrant its merit against some set of standards, 
after the evaluand has been developed into Its putatively final 
form. Fonnative worth evaluation is performed to facilitate adoption, 
adaptation, or fitting of the evaluation to some local context of 
use, Summative Worth elraluation is performed to warrant or 
certify an evaluand for permanent local (situational) use. 

Based on this analysis the following definition for the activity 
of evaluation is proffered: 

EVALUATION is a type of DISCIPLINED INQUIRY undertaken 
to determine the VALUE (MERIT AND/OR WORTH) of some 
entity— th6 EVALUAND— such as a TREATMENT, PROGRAM, 
FACILITY, PERFORMANCE, and the like--in order to IMPROVE 
OR REFIN^'the evaluand (FORMATIVE evaluation) or to 
ASSESS ITS IMPACT (SUMMATIVE evafMation). 

This definition asserts that evaluation, like research, is a type of 

disciplined Inquiry, and should therefore exhibit the properties 

12 
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prescribed for such Inquiries. It Is Intended to establish merit 
and/or worth (Guba and Lincoln, 1981). The evalcand is described 
as a treatment, program, facility, performance, or similar entity; 
it is not (looking ahead) a policy option. Evaluands may be assessed 
formatively or summatively, depending on the purpose of the 
evaluation. 

This definition appears to be acceptable on the premises of 
either the conventional or the naturalistic paradigm. Advocates of 
neither would quarrel with the cieim that evaluation is (or can be) 
disciplined inquiry. That evaluation deals with the determination 
v - value cannot be an Issue either, since the very term evaluation 
has value as Its root. The identification of two kinds of value— merit 
and worth — can only be regarded as a useful distinction (although, 
to be sure, some might think It superfluous, a matter of overkill. 
But that judgment would not be made on the basis of the judge's 
paradigm affiliation). That evaluands should be described as 
treatments, programs, facilities, or performances (or similar entities) 
is also not unusual, although the naturalist might be somewhat 
offended by "treatments" since that formulation seem to play into 
the hands of those who would regard evaluation (ideally) as a form 
of experiment. The distinctions between formative and summative 
evaluation are also well understood and universally accepted by the 
profession. 

What is of special interest here Is the delineation of those 
features that distinguish evaluation from research. First and 
foremost v.e must be struck by the difference in purpose : research 
is undertaken to resolve some problem, while evaluation is under- 
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taken to establish value, jon that score alone research and evaluation 
are monumentatiy different, and this fundamental difference reflects 
itself in the products that result, In the expected outcomes or 
uses of the inquiry, and in the audiences to whom products are 
addressed. • 

Research Is typically adequately served by a technical report 
(which may be a book monograph, or journal article), but technical 
evaluation reports are rarely sufficient to meet the needs of, or 
communicate well with, the va**tety of stakeholding audiences. 
Reports which can be utilized by many different audiences often 
take the form of case studies. Additional, less formal, but also 
more responsive reporting formats may also be utilized as audience 
needs dictate: an oral report, sl?de show, filmstrip, town meeting, 
special program on local television or Tdi6\o, and the like. The 
format of most evaluation products, with the possible exception of 
summative merit evaluations which address audiences outside the 
local context and therefore probably need to be written, is largely 
determined by needs of local community groups or users, and 
depends in part on their ability to deal with varying degrees of 
complex technical material. 

While the expected outcome or use of research is simply the 
expansion of understanding (wnether in the erklarung or verstehen 
sense), the outcomes and/or uses of evaluation Inquiries vary 
depending on the kind of evaluation which Is undertaken. For 
example, the expected outcome of a formative merit evaluation will 
be modifications, improvements, or refinements in the design of the 
evaluand (treatment, program, facility, performance, and the 
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like). The expected outcome of a summatlve merit evaluation will 
be the certification or warranting on the evaluand as intrinsically 
valuable, meriting consideration for adoption or use in some setting 
other than the one in which It was developed. The expected 
outcome of a formative worth evaluation will be adaptation of meri- 
torious evaluand to a particular local setting, and the expected 
outcome of a summatlve worth evaluation is the certification of such 
an evaluand for extended and long-term local use. 

Whfie the audiences for a research inquiry may be primarily 
peers of the inquirer (other scientists, say), the audiences for an 
evaluation Include all of the many stakeholders who may have some 
investment in the evaluand. These include (at a minimum)' develop- 
ment teams that have been involved in designing and developing 
the evaluand; groups of potential local adopters; a local adaptation 
team charged with "fitting" an evaluand into the local context; 
groups of local decision makers who must decide what actions to 
take in regard to the evaluand; and, finally, groups of beneficiaries 
( e.g. , clients, target groups, their parents or spouses, and the 
like) and of potential victims (e.g. , persons whose interests were 
subordinated to free resources so thaV the evaluand could be 
developed and/or implemented). The stakeholders will differ from 
locale to locale. While the audiences for a research Inquiry are 
relatively homogeneous, the audiences for evaluations are multiple 
and heterogeneous, a fact which, at the very least, introduces 

m 

political considerations to an extent simply not found in research 
(although the fact that research is also value-bounded should not 
be overlooked). In all events, the purpose of a given form of 

15 
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evaluation dictates who ought to be interested (a normative statement)^ 
but not who finally wiH be (an empirical and political matter). 

Policy Analysis 

Consensus on a definition of policy ancilysis is no more in 
evidence than in th*^ case of research or evaluation. Yet there are 
sufficient similarities amono definitions proferred by different 
writers that it seems li^ jy that members of the social science 
community are opera^' ;g along parallel lines. Two examples should 
suffice to make the point. Nagei (1979) defines policy analysis as 
"the how-to-do-it methods associated with determining the nature, 
causes, and effects of governmental decisions or policies designed 
to cope with specific social problems" (p. 7). Nagel terms '^Is 
activity "policy analysis research"— a further case of legitimation. 
Peter House, In a work entitled The Art of Public Policy Analysis 
(1982— a title which suggests that policy ana'ysis is something 
other than a science)— carefully skirts the issue of definition in 
his first chapter, but notes In a footnote at the end of the chapter: 
"I shall use the terms policy analysis, systems analysis, and 
operations research interchangeably, since I believe that the need 
for formalism and analytic techniques is similar to all, as is the 
underlying system structure and response" (p. 53). 

A critique of constructions . Nagel's definition focuses primarily 
on method, to the subordination of impact, and is focussed rather 
more on analyzing policies-in-action than on projecting the impact 
of proposed policies (which we call policies-ln-intent).^ House's 
interchangeable terminology Is also unsatisfactory; it can be only a 
supremely conventional thinker who would lump policy analysis. 
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systems analysis, and operations research within the same rubric. 

Both operations research and systems analysis have embedded in 

them assumptions which are not only relatively unuseful but to 

some extent destructive of good policy analysis: the assumption of 

linear causality (embedded in the word "syscems** with its mechanistic 

metaphor of parts of a great machine all moving in concert), the 

assumption of value*freedom (defined by House's insistence on 

rigorous scientific method and objectivity), and the heavy emphasis 

on formal i Stic aspects of tl-ie process, with its Inherent assumption 

of generalizability from analysis to analysis, and the concomitant 

need to "discover order and structure . . . [as] as feature of the 

sci'^ntific approach" (p. 240). 

Nevertheless both Nagei and House have major contributions 

to make. Nagel suggests that policy analysts are "involved mainly 

in determining the effects of alternative public policies" (and 

indeed, this statement is a better definition of policy analysis than 

his earlier one cited above), and suggests that they perform this 

function in any of four ways, by 

. . . (1) taking policies as givens and attempting to determine 
what causes them; [or J ... (2) taking social forces as 
givens and attempting ... (3) taking policies as givens and 
attempting to determine what effects they have; [orj (4) 
taking effects as givens and attempting to determine what 
policies will achieve or maximize those goals (1979, pp. 7-8). 

Nagel characterized the third approach as "evaluation research" 

and the fourth as the^ "optimizing perspective," and declared that 

the latter would serve as the focus for his book, which will deal 

mainly with determining the outcomes or effects of different or 

alternative social policies. 
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>tjng the optimizing perspective has much to commend it, 
since it fo^usse^ on the acceptance of broad social mandates and 
concentrates on achieving those, of as much/many of them as 
possible, with optimal use of available resources (a phrase which 
suggests some mix of manpower, money, time, and other means). 
Another characteristic of Nagel's definition which seem to be sound 
is the forthright role to which he assigns values. Me is quite ^ 
clear that "policy analysts cannot be totally value-free, since they 
are seeking to achieve or maximize given values, but they can take 
extra precautions to keep social or personal values from interfering 
with their statements of fact " (1979, p. 9, emphases added). 
However, in describing what constitutes good policy analysis (a 
presumed outcome), he suggests that it ought to be "empirically 
valid, in tfte sense of conforming to reality" (p. 10). This state- 
ment is not only at odds with the emergent-paradigm or naturalist 
position dh ontology but takes a very synoptic view of policy as 
singular, conforming to the view that reality is singular (when 
there is "real" reality there can be only one policy that conforms 
to it exactly). Naturalists would of course assert that policy 
ought to be viewed as multiple in form, in line with the proposition 
that realities are multiple. 

In the case of policy, multiple realities can take at least three 
forms: the policy-in-tntention , the policy-ln-action , and the 
policy-in-experience . These three forms may be conceptualized 
respectively as statements about policy, or the policy as constructed 
and written down; activities and behaviors that are displayed by 
agents in process of Implementing policy (including the process of 
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local adaptation and/or diversion as practiced, probably necessarily, 
by street-level bureaucrats, I.e., the agents in face-to-face contact 
with the client or target group); and the experiences of the client 
or target group as they receive the policy (Guba, 1984), That 
policy may be viewed as three different processes/products by 
three different audiences betokens conducting analyses that Jreat 
it as potentially three different social realities (at least). Thus, 
Nagel's criterion of empirical validity is most assuredly one which, 
while supportable, does not go nearly far enough. Empirical 
validity begins and ends with the typical social science construction 
that policy, like reality. Is unitary and synoptic, when it is clearly 
not, even to social scientists. 

Nagel also suggests that the results of policy analyses should 
provide "good insights which are clearly communicated" (p. 10). 
No one would disagree with this conclusion, but what remains at 
issue is to whom, under what circumstances, and how such analyses 
should be communicated. Assuming a unitary character for policy 
makes it virtually certain that its insights will bo largely wasted, 
since they are unlikely to be shareable with (or understandable 
by) the variety of audiences wTio may be concerned. It seems 
quite likely, in fact, that those who most need good policy analyses- 
clients and target groups--will be among those least likely to hear 
the results. 

Finally, Nagel goes on to say that policy analyses ought to be 
conducted In such a way that they produce "desired social conse- 
quences," still another synoptic prescription. Recognizing that it 
is becoming increasingly difficult to satisfy everyone In a pluralistic 

/ 
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society, the idea of desired social consequences is loaded with the 
freight of consensual decision-making, value consensus, and majority 
rule. Consensual decision-making rarely ^occurs, since most decisions 
in the public policy arena are constructions resulting from trade-offs, 
compromises, negotiations, and politicaf concessions- -a perfect 
example, in fact, of mutual simultaneous shaping. There is nothing 
inherently wrong with such political development, but one must 
recognize that the political negotiation is a continuously ongoing 
process. Thus, the policy which was intended (by Congress, say) 
often turns out not to be the policy which is written (by the 
Federal administering agency, say), or the policy adapted in the 
process of devising the rules and regulations which accompany its 
promulgation. Likewise, assuming a consensus of opinion ignores 
the fact that policies which seek to enable or create advantage for 
some group viewed as disadvantaged may redress their social ills 
at the expense of some other group. Finally, it is becoming 
increasingly clear that majority rule applies only in rarified situations 
(not to include Presidential elections, wnere the majority of eligible 
voters do not exercise their right); in most situations, pluralism is 
increasingly respected, and the protection and encouragement of 
minority expression (whether political, racial, ethnic, > linguistic, or 
cultural) has become the rule. As a result, desired social conse- 
quences change frcmi locale to locale and context to context. The 
function of policy becomes more a matter of ensuring fair treatment 
to all rather than determining how best to achieve some mutually 
desired outcomes. 
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Fair treatment is the equitable distribution of justice as 
determined within the framework of context-relevant decisions. 
The political process is reconstructed not as majority rule, but aw 
push-pull fine social tuning. Groups give voice to needs which 
may or may not be met depending on likely benefits, projected 
costs, certainty of returns, and legal and political constraints. 
Desired social consequences change. And with those changes 
comes a continuing necessity to re-establish what may be desirable 
at any given time and place. 

Returning U> House's (1982) formulations, we see that he 
makes contributions which help non-political scientists to understand 
what he believes a policy analyst is (another way of stating what a 
policy analysis ought to be, by looking at what policy analysts 
do). Using a table outlined by Nagel (1980), House has constructed 
a typology of the kinds of policy analysts that exist (scientific, 
professional, political administrative, and personal); how they 
separately view public policy problems (for examp'e, the scientist 
sees them as theoretic problems, the professional as design problems, 
the political analyst as vaiue-maximization problems, and so forth); 
and the motivations that drive each of the five types (for example, 
the search for theory, regularities, and "truth"; improvement of 
policies and policy-making; advocacy of policy positions, effective 
and efficient policy implementation; and concern for policy impacts 

'^on life). He goes on to suggest some poHiblfi_appt^achei,-to^ — 

policy analysis and some means of training persons who are to 
carry them out, since often the policy analyst operates in a different 
sphere from traditional, university-trained and based social scientist. 
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Concentrating on the tradition (which he labels relatively 
recent), the methods / the timing ("the policy analyst normally 
operates under a restrictive tyranny of time/' p. 27),, the resources , 
the personnel constraints , the quality ("maintaining objectivity in 
the course of doing an analysis is a constant problem for policy 

c 

staff," p. 29), and the clients .^ House argues that the policy 
analyst Is really a very different person from the typical- -and 
typically more conservative— social scientist. While ttie latter 
expects to be in control of problem, data, and technique, the 
policy analyst expects no such luxury. The comfort of certainty 
which the scientist (putatively) possesses Is unknown to the analyst, 
who operates under great uncertainty and with time reistraints 
which are unknown In normal Inquiry. 

House has failed to note, however, that while the expected 
products and audiences are very different, most policy analysts 
now operating are trained in exactly the same tradition as scientists 
and by persons w'lo train social scientists in general. Arguments 
regarding timing, personnel, resources, and the^Uke aside, most 
policy analysts function exactly as their mentors- -other social 
scientists— have trained them. As a result, they sTYi^ro not only 
methodological biases, but paradigms and philosophical persuasions 
as well. While policy analysts operate under very different circum- 
stances and often iri very different surroundings from most social 
scientists, they nevertheless emerge from an atmosphere which 
House himself tellingly described: 

Science has its own training grounds, called universities; 
...it inculcates Its disciples with the beliefs of its tradition'; 
...the teachers and professors in these Institutions are priests 
and the students are the faithful. The purpose of these 
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institutions is often to move the novitiates through the indoc- 
trination process... (1982, p. 23). 

it is hard to imagine these methodological and paradigmatic apples 

falling very far from their trees. While House argues that the 

policy analyst is a very different breed of cat, it se^s unlikely, 

on the basis of House's own analysis, that they function in ways 

other than as traditional social scientists, trained in traditional 

social science programs and paradigms. f 

Naget and House have been among the most persuasive of 

extant writers, and their premises look very much like those of 

other social scientists (see, for example, Carley, 1980; Jenkins, 

1978; Lasswell, 1971; Lindblom, 1959; Ripley, 1975; Schulman, 

1975; Stokey and Zeckhauser, 1978; and Wildavsky, 1979). The 

one labels policy analysis as research; the other disclaims the old 

traditions of science, but puts policy analysis squarely in the same 

category with operations research, systems analysis, and the old 

mold . of the social scientist. The point of this brief excursion is 

If 

that we remain an adequate, satisfactory or consensual definitions 
of policy analysis. 

Toward a functional definition and sphere for policy analysis . 
Gub^ (1984) believes that there may be as many as eight different 
constructions of the word tx)licy in the literature. These range 
from the statement that "policy i' an assertion of intents or goals" 
to "policy is a strategy undertaken to solve or ameliorate some 
problem" to "policy is the impact of the F>ollcy-making and policy- 
implementing system as it is experienced by the client" (pp. 64-65). 
"It is nonsense," he asserts* "to ask the question, 'What is the 
real definition of policy?'" Since "all . . . definitions are con- 
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structions ; none can claim tangible reality . . . [and] virtually 
any policy definition must be admitted sc long a$ its proposer can 
make a rational case-for his or her particular usage" (p. 70). 
What is crucial about the definition of policy that one accepts is 
that 

. . . not all definitions are equal in their consequences for 
policy analysis .... Each definition calls for its own datd/ 
sources, and methods^ and produces unique outcomes . . . 
(and) each different definition has an enormous impact on the 
processes and products of policy analysis (p. 70). 

Whatever the definition of policy that may be adopted, it should fit 

the purposes that the analyst has in mind, that Is, there must be 

resonance between definition and purpose. 

Coleman, according to Car ley (1980), has proposed still other 

"important distinguishing characteristics ..." which expand on 

the purpo^ies, the nature and flttingness of policy analysis as it is 

carried out: 

1) the audience is a set of political actors, ranging from a 
single client to a whole populace, and the research [sic] 
is designed as a guide to action; 

2) partial information available at the time an action must be 
taken is better than complete information after that time; 

3) the criteria of parsimony and elegance that apply in 
disciplined research are not important; the correctness 
of the predictions or results is important; 

4) the ultimate product is not a "contribution to existing 
knowledge" in the literature, but a social policy modified 
by the research (sicj results; (and the understanding 
that] 

5) it Is necessary to treat differently policy variables which 
are subject to policy manipulation, and situational variables 
which are not (Coleman, cited In Carley, 1980, pp. 
25-26). 

The characteristics most of note here include the emphasif. on 
different sets of political actors, including clients; the general 
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worthlessness of parsimony and elegance, which usually are thought 
to characterize. good research and theory development; the contribu- 
tions to modifying', altering or otherwise re-structuring social 
policies; and finally, the recognition that some variables (which we 
would call elements or factors or patterns) aie subject to manipulation 
or alteration, and that others, which Coleman terms situational, 
variables (and which we would call contextual elements) are not. 
In fact, Coleman's discussion greatly amplifies some of the problems 
which have been associated with more traditional definitions of 
policy analysis as a form of research, especially those connected to 
and with the positivist paradigm, with its emphasis on causality 
which is linear, and a value-free objectivity which is impossible to 



The characteristics which Coleman proffers are used to mount 

a trenchant criticism of policy analysis as it Is supposedly practiced 

by the analysts (Carley, 1980): 

One could easily argue that most public sector decision-making 
is in the end the result of a political bargaining process. 
This being the case rational analysis carried on in an ignorance 
of political reality may well end up so divorced from social 
reality as to be of little use to anyone. By the same token, 
however, vague and unsystematic "political** research loaded 
with implicit causality and value judgments, and not subject to 
exposure or dissection , is of no great value to policy making 
either. A balanced perspective helps policy makers and 
researchers select criteria for judging the relevance of analysis 
to a particular policy problem. It does this by encouraging 
examination of the divergency between the problem as defined 
by the policy maker and as by the analysts, and by 

arguing that no analysis is understood until It is clear~what , 
and whose, value judgments are part of the analysis— value 
judgments which must be considered an integral part o^ every 
analysis (pp. 6-7; emphases added )." 

These statements eloquently express the differences between 



policy analysis and research (except for Colelnan's unfortunate use 



attain. 
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of the term research itself). The emphasis on multiple audiences, 
the willingness to accept partial fnfonpatiofi in the interests of 
serving irtiportant decisions, the foreknowledge that "messy" may , 
be better In some instances than "elegant," the insight that policy 
analyses are performed for the sake pf critical action, not advance- 
ment of nomothetic knowledge, and the recognition of the differences 
betw^n action options and contextual givens resonates well ind^^ 

* .r 

with the emergent of naturalistic paradigm. 

Given thf" earlier definitions for what constitutes disciplined 
Inquiry, and. with the assertion that policies are constructions , 
growing- but of multiple realities and multiple levels of ejtperiencing 
^e policy, the definition, In the literature wbicii comes closest to 
our understanding- is tfrwt suggested by Ukeles (1977): policy 
analysis is "the systematic investigation of alternative policy options 
and the gatifering and , display oC^ evidence for and against each 
option" (f/ 224). The emphasis on systematic Investigation,' on 
multiple policy options, on the^gatherlng of evidence (with reliance 
on empirical evidence), and the dUplay of/that evidence seem to 
us particularly appropriate. 

But we are stHI left wfth discomfort, since no definition fully 
satisfies ^^\ desirable criteria. That problem can be solved by a 
new proposed deflnitioh: * ^ 

POLICY ANALYSIS Is a type of DISCIPLINED INQUIRY uv\6vir' 
taken to GATHER and DISPLAY EVIDENCE (including CONTEX- 
TUAL DATA) for and against At^ERNATJVE POLICY OPTIONS 
(INTENDED, ALREADY IMPL^^JI^ o/eXPERIENCED) in 
order toJNFORM ^EjGOTIATIONS OV¥R CHOICES in terms of 
the MULTIPLE. VALU^ of RELEVANT AUDIENCES.* 

'.What are. the implications ^this definition? First, policy 

analysis Is sald^ be'pfirect?d to multiple audiences. For policy 
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makers / ^ policy analysis \s concerned with policy in-intentlon , 
that is, the intended achievements of the proposed policy options, 
with concomitant analysis of what the expected and unexpected 
outcomes might be (the outfcomes-plus-unexpected fallout construction). 
For those charged with carrying out the policy throughout the 
various levels of the government of other policy-implementing 

> 

agency / the analysis will address pol icy- in- implementation , that is, 
the policy as it migfit reasonably be carried out In adapted form in 
a variety of contexts (note that one policy-in-lntention may. spawn 
multiple policies-in-implementation). For some analyses this coujd / . 
mean', for example, lookinp at the effects of policy Implementation 
of brock grants from state to state, or within a specific state. 
Poiicies-in-intention Invariably are shrunk, stretched, nipped, 
tucked, and otherwise tailored to fit what are seen to be local 
needs (including the needs of the implementers as well!). The 
policy-in-intention isr once removed from the original vision (a . 

political accommodation?) in the policy-makers' minds as it emerges 

• • 

into the policy-in-implementation stage.. . * 

At two removes is the policy-in-expepienoe ; the logical audience 
for an analysis at this level is the client o'* target group(s) envisaged. 
Such analyses are defected at enhancing receipt of service, considering 
alternative options for resource allocation (Is the wav the resources 
are allocated actually meeting the greatest needs of the largest 
proportion of the client group?), clarifying the delivery mode (Can 
clients find their way through the delivery system?), and for 
refining interpretation (Is this policy providing for the most salient 
needs?) (Guba, 19P<i). 
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Second, good policy analysis facilities choice, and choice is 
value'tnediated through political negotiation. The value judgments 
must be clear and publicly ihspectabie (to qualify as disciplined 
inquiry), and must be tied to the multiple value perspectives of 
client or target groups as well as other relevant audiences. 
Carley (1980) implies that this criterion can be met if options are 
defined by both policy makers and analysts in a negotiated or 
collaborative way, but it should be clear that the client or target 
groups ought also to be involved in this definitictai task. In 
proposing the delivery of services, benefits, goods, or other aims 
of public policy, the values of client groups must be explored and 
arrayed as carefully as those of policy makers and analysts. 

Third, jthe definition requires that contextual variables be 
taken into account. In the recognition of those things that are 
subject to change (what Coleman calls policy variables) and those 
things which are not amenable to change (what Coleman calls 
situational variables) lies the roots of grounding , that is, embedding 
decisions in the data of local contexts. The appreciation of the 
stability of local conditions makes for sophisticated analyses, while 
the assumption of manipulability of local contexts makes for awkward 
and sometimes perverse adaptation of public policy. 

Fourth, the definition highlights the negotiation processes 
which can, do, and should occur in policy making and implementation. 
This provision is both pragmatic . In that policies ought not to be 
designed unless they meet needs that have been identified with the 
aid of the client group, and ethical , in that individuals and groups 
have the right to self-determination and to control over major 
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portions of their lives. Negotiations between policy makers, analysts, 
and clients represent appropriate, equitable, fair, and respectful 
ways in which to proceed in building policy. Policy developed 
without consultation with the client (without the consent of the 
governed, as it were) fs equivalent to research conducted without 
negotiation and reciprocity, but elong the traditional "take what 
you need and get out^ lines (what Shulamit Reinharz, 1979, calls 
the "rape" model of research). Policy built in this way stands a 
good chance of being ineffective, misdirected, badly received, and 
otherwise wasteful of resources. 

Finally, the definition suggests that the results of a policy 
analysis must be moved into arenas appropriate for display and 
debate. The best analyses cannot be effective if they are not 
provided to appropriate audiences for argument and contention. 
Carley (1980, p. 30) concludes that in many instances it is perfectly 
appropriate for policy analysts to "concentrate effort on outlining 
broad alternatives and elucidating the value choices and the value 
sets of participants relevant to the problem." The pur^^se of 
policy analysis is, after all^ to illuminate choices, not to close them 
out. 

In that sense the group of social scientists that seems to have 
done the best job of outlining broad alternatives to the public in 
recent years fs the economists. Regular columns in each of the 
national news magazines, sophisticated debate on Issues/talk television, 
and constant coverage by the news media have brought economic 
issues to the fore. Competing perspectives and competing theories 
on how Western €KU>nomies might reduce Inflation, narrow national 



29 



27 



deficits^ encourage productivity/ fight recession ^ bolster personal 
savings^ and the like; have brought economic issues from the 
universities and the Federal Reserve to the living rooms of American • 
consumers. Economic theory doesn't seem to offer much In the 
way of solutions, but the voter unaware cf the term Reaganomics 
or unfamiliar with the basic principles (if not the name) of Keynesian 
economics is rare. As it turns out, the average citizen has little 
control over the Federal Reserve, or over interest ral*3S, so the 
debate is one in which he or she can participate only indirectly, 
but economics serves as an examplar for how policy issues might 
be brought to stakeholders and audiences by those who design and 
legislate policy. 

Policy analysis also differs from both research and evaluation 
as a mode of disciplined inquiry. Research yields technical report; 
and evaluation assessments of merit and worth; policy analyses 
yield different outcomes not only from these but also from one 
another depending on which definition of policy is adopted (Guba, 
1984). So for Instance, if the operational definition of policy Is 
"an assertion of intents or goals ," then a policy analysis yields a 
prioritization of goals to be achieved. If policy is a "governing 
body's 'standing decisions' by which It regulates, controls, promotes , 
services, and otherwise Influences matters within its sphere of authority , 
then a policy analysis yields a set of rules, as for example, that 
parents must be afforded due process before having their children 
placed In special education classes. If policy is a guide to dis c retionary 
action , then policy analysis Identifies key action roles or recommen- 
dations for discretionary limits. For example, in the university 
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accreditation process, evaluitor-consultants may recommend non- 
accreditation if they believe the institution will be unable to provide 
funds for a viable program in the future, but those institutions 
may appeaf if they believe that their situation has been misassessed 
or misrepresented. If policy is "a strategy undertaken to solve or 
ameliorate slome problem ," then policy analysis Identifies common, 
special, or recurrent problems, and the development and exploration 
of sets of strategies for dealing with each. If policy is "sanctioned 
behavior , formally through authoritative decisions, or informally 
through expectations and acceptance, established over (or sanctified 
by) time," then policy analysis provides a set of role definitions 
and corresponding expectations, for example, that regular c!assroom 
teachers shouki consult with a resource teacher or a school psychol- 
ogist when they are unsure about how to handle a case involving a 
special education student. If policy Is a "norm of conduct, charac - 
terlzec by consistency and regularity , in some action area ," then a 
policy analysis results in descriptions of satisfying behavior and 
corresponding norms, for example, that teachers with malnstreamed 
children in their classrooms ought to Keep paperwork sufficiently 
up'to-date to satisfy a state department monitor who might wish to 
inspect the Individualized educational plans (lEPs). If policy is 
defined as "the output of the policy-making systems : the cumulative 
effect of all the actions, decisions, and behaviors of the millions of 
people who work in bureaucracies; [or] an analytic category," 
then policy analyses produce descriptions of collective efforts of 
entire systems or bureaucracies. Finally, if policy is "the impact 
of the policy-making and policy-implementing system as it Is ex - 
perienced by the client ," the chief policy analysis products are 
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client or target group constructions (positive and negative) of 

what the policy Is doing in their lives (Guba, 1984).. Guba's 

earlier (1984) analysis provides a set of handles by which the 

different definitions may be grasped: 

In the case of Definition 1, policy "looks like" a set of 
ends . Definition 2 results in policy statements which look 
like ruleS / while definition 3 results in guidelines with some 
built-in discretion. Definition 4 results in a set of tactics . 
Definition '5 results in expectations / 6 in norms , 1 in effects , 
and 8 in encounters (p. 6; the preceding paragraphs are 
adapted freely from pp. 8-15; italics in original). 

To recall the earlier discussion, definitions 2 through 4 can be 

described as relating to pollcy-in-intention , definitions 4 through 7 

to poiicy-in^implementation , and definition 8 to policy-in-experience 



VW^Ie the categories are not entirely exclusive, stakeholding 
sudiences for the policy products corresponding to these different 
definitions will be policy-makers for policies-in-intention (although' 
client groups have had and will continue to have, through advocacy 
groups, local and national level hearings, and court decisions, 
impact on intended policy); policy implementers for the policy-in- 
impiementation definitions; and those who experience policy--client 
and target groups--for the policy-in-experience definition. 



To the question of whether research, evaluation and policy 
analyses are distinguishable the answer is yes, clearly each of 
these activities fits comfortably within the definition of disciplined 
inquiry (Cronbach and Suppes, 1%9). But by the definitions 
proposed, and from a consideration of their implications, all three 
are unique activities. These definitions do not differ materially 



(pp. 64-65)^ ' 




Are There Differences Without Distinctions? 
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from a consensus or combination of those ccMnmonly found in the 
literature, but they are stated in language that can be easily 
assimilated within either of the major paradigms, positivist or 
naturalist. 

Systematic differences in Intention or purpose of the activities 
signal profound differences in expected products, outcomes, and 
intended audiences. The distinctions drawn dramatize the need to 
view research, ev aluation, and policy analysis^as separate, discrete, 
and mutually exclusive activities, if that is the case, then research 
is research, evaluation is not evaluation research but simply evaluation, 
and pcllcy analysis is neither evaluation in another form nor is it 
policy analysis research. It is, simply, policy analysis, although 
the tools, methods, and approaches may be similar, particularly 
during data collection and analysis phases. Correspondence occurs 
only to the extent that techniques used to arrive at data, and to 
draw conclusions from them, may occasionally overlap. And while 
methods may be useful to distinguish some paradigmatic choices, 
they are neutral with respect to the type of disciplined inquiry in 
whose behalf they i^ay be utilized. 

Does the ^hoice of Paradigm Matter ? 
The matter of paradigtpatlc differences cannot be overlooked, 
it is becoming increasingly ^lear that the dictum "believing is 
seeing** applies in all three activities; what we hope or expect to 
see blinds researchers, evaluators, and policy analysts to other 
questions or issues which may be of pressing,, or at least of equal, 
import. Decisions about how to label an activity are important but 
the prior decision regarding the philosophical and epistemoiogicai 

33 



31 



Stance which will guide the inquiry is equally crucial. The current 
competition between inquiry paradigms in the social science community 
is founded on a fundamental disagreement about what we shall 
"see" (or what we should persuade our students to be able to 
see). So the question of what paradigm should guide disciplined 
inquiry moves to the forefront of the debate. 

Elsewhere (Cuba and Lincoln, 1981; Lincoln and Cuba, in 
process) we have made the argument that the emergent naturalistic 
paradigm is more useful and demonstrates greater power in inquiry, 
particularly social/behavioral inquiry, because its assumptions 
provide a better fit to phenomena of interest (especially as we 
have come to know them lately), because it is more resonant with 
emergent substantive paradigms in fields ranging from basic physics 
to brain theory (especially as delineated in the work of Schwartz 

and Ogiivy, 198 ), and because it has been able to deal better 

with the crises (in the Kuhnian sense) that have afflicted the 
conventional, positivistic paradigm beginning about the turn of the 
century. Since research, evaluation, and policy analysis can all 
legitimately be viewed as exemplars of disciplined Inquiry ail of 
these observations apply to each of them. Insofar as more traditional 



definitions of these three inquiry forms have tended to exhibit bias 
in favor of the conventional paradigm, we have proposed new 



finitlons which open the possibility of applying the new paradigm 



frame the definitions In terms that would not close out older options 
for those who elect to stay with them. 




with equal legitimation, although we have been at some pains to 



34 



32 

In addition to these compelling arguments, we havi^ voiced 
some additional considerations in favor of the propos'tion that the 
naturalistic paradigm should be the paradigm of choice. With 
respect to research, we are prepared to argue that use of the 
naturalistic paradigm obviates the basic/applied dilemma. If It is 
the case that all inquiry is in some sense conte>ct^boun<^, because 
phenomena take their meaning from tbeir contexts as much as from 
any intrinsic characteristics, then all phenomenolQgicdl studies are 
applied. If we cannot hope for generalisations but only for working 
hypotheses, the very idea of '*basic" research is shaken to its 
roots. If there is no linear causality, the hope for a nomothetic 
social science is largely vain. 

But even If one wished to maintain the basic-applied distinction, 
the argument in favor of the naturalistic paradigm can still be 
made. For naturalistic inquiry facilitates theory development 
(through the discovery of grounded theory) and provides the 
"thick description" in terms of which other researchers can further 
test hypotheses or determine the limits of transferability. In 
relation to applied research/ the question of application— to a 
particular situation at a particular time and pldce*-is central. 
Naturalistic inquiry is in the best position to determine those 
contextual el^ents that support the design and development 
process, or which need to be considered before an application can 
be made. 

> 

With respect to evaluation, we have made the case in detail in 
our earlier work (Guba and Lincoln, 1981) that the naturalistic 
paradigm Is the paradigm of choice, at least insofar as one takes 
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responsive evaluation as the focus for the evaluation. To be 
responsive implies the identification of stakeholding audiences«'which 
can only be done in context--and of interacting with those audiences 
to discover their claims or concerns with respect to the evaluand 
and the issues they may wish to raise about it. It is immediately 
evident that the conventional paradigm can test only those claims, 
concerns, and issues which the evaluator is in a position to foresee; 
others will necessarily \k> t>y the board (as Scriven so cogently 
points out In his discussion of "unintended" effects). Further, 
insofar as the evaluation may be concerned with worth (as most 
evaluations are), it must be conducted in the context of application. 
Evaluation rests on the valuing process, so that the values (per- 
spectives) of different audiences must be taken into account. 
Since one cannot count on value consensus (indeed, value pluralism 
is today's mode), evaluation must culminate in a negotiation process. 
Good evaluations, as is coming to be more and more appreciated, 
cannot end with a simple report of conclusions and reccNnmendations 
but in the delineation of value judgment options which must be 
further explored before any action alternative can be specified. 
Thus the evaluator becomes less technician than change agent; less 
objective scientist than active collaborator. All of these considera- 
tions are clearly more resonant with the naturalistic than the 
conventional paradigm. 

Many of the same observations also apply to the policy analysis 
arena. Policies are never singular; policy options represent different 
realities and may exist at different levels (Intention, implementation, 
and experience). Policy decisions are the culmination of a political 
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negotiation and bargaining process. Policies emerge as the outcome 
of a series of mutual shapings, which cannot be understood except 
as a whole. Value pluralism also characterizes the policy arena. 
Different contextual factors argue for different policy formulations. 

When it is policy-in-intention whiclY is considered, a policy 
analysis completed under naturalistic assumptions will facilitate 
determination of both pollcy-maker intentions and client needs. 
When It is policy-in-implementation which is addressed, naturalistic 
inquiry facilitates the discovery and description of informal (and 
occasionally formal) adaptations made by those charged with imple- 
mentation ("street-level bureaucrats")* Delineation of discretion as 
exercised by these agents is still largely underdescribed; we know 
very little about the kinds of discretion that are exercised and 
still less about the differences in latitude that exist (or are exploited) 
between the policy as intended and as implemented. Naturalistic 
inquiry at this level of policy analysis would bestow insight into 
the process of local, imprcmiptu decision-making as it is carried out 
with respect to the clients or targets of policy. Finally, we have 
virtually no studies which document and illustrate the policy-In- 
experience dimension. Policy studies have tended to be top-down, 
ethogeneous inquiries which tell about cost-benefit ratios (or more 
likely, which advance reasons why such ratios cannot be computed), 
about populations, income levels, neighborhood housing patterns, 
and the like, but they rarely aid us in seeing how clients experience 
the^policy as enacted (as for example, that many Blacks feel the 
policy of informing them about sickle-cell anemia is simply a genocidal 
plot). Naturalistic inquiry provides the basis for vicariously 
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experiencing what the clients encounter as they attempt to negotiate 
the bureaucratic structure. Studies of this latter type would 
clearly have a major impact on altering norms of service delivery 
to affect groups. 

It is our argument then that use of the naturalistic paradigm 
in all three inquiry areas— research, evaluation, and policy analysis- 
is both appropriate and warranted. While these areas differ in 
important ways from one another, as we have tried to show, paradig 
maticaiiy they are all more amenable to providing meaningful results 
in inquiry when it is carried out naturslistically than conventionally. 
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Footnotes 



* See, for example, Ltncoin, Y.S., "The structure of promotion 
and tenure decisions In Institutions of higher education/' Review 

Higher Education , 6 (Spring, 1983), 217-232, for a larger 
discussion of this example . 

2 The meaning of the terms policy-in-action and pollcy-in-intent 
will become clearer shortly. It is our contention that policies lead 
three lives: those which represent their intent , those which 
represent their implementation , particularly by "street-level bureau- 
crats (Lipsky, 1981), and those which represent the policy as 
experienced by Its clients or targets. Thus there is a major 
difference between what a policy is intended to do, what it does, * 
and what it is experienced as doing. / See below, and Guba, 1984. 

^House implies that the clientele for a given analysis is singular, 
or at least highly convergent, since "often the content of a policy 
analysis is influenced greatly by the Known or anticipated desires 
and demands of ^e decision-mailer fort whom it is being prepared." 
See p. 29. 

* The reader should note that a policy option is not a treatment, 
program, facility, performance, or anything similar; these are 
properly objects for evaluation and repres^t scMiiething done in the 
name of a policy. For example, negative income tax--a treatment— Is 
put in place in the name of the policy of ameliorating the condition 
of the indigent (another treatment that might have been nwunted 

in the name of that same policy is called welfare). 
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